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(5^1) rule: IMPROVED METHOD AND APPARATUS FOR CREATING VIRTUAL WORLDS 




(57) Abstract 

A computer display displays a data flow network as a plurality of interconnected units. An interactive program allows the 
appearance of the plurality of interconneaed units to be changed. The data flow network then is automatically altered to corre- 
spond with the visual changes. 
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yyipROVED METHOD AND APPARATUS F OR CREATTMG VTRTOAT. WnPJ .nc; 

5 BACKGROUND OF THE INVENTION 

This invention relates to virtual reality systems 
and, more particularly, to a method and apparatus for 
creating a virtual world. 

Users of computer systems are now able to create 
10 virtual realities which they may view and interact with, 
one type of virtual reality system is disclosed in U.S. 
patent application No. 535,253, filed June 7, 1990, entitled 
"Virtual Reality Network," the disclosure of which is 
incorporated herein by reference, one task which must be 
15 performed is the creation of the virtual worlds within which 
the users interact. This can be a very tedious task, since 
a complete description of all virtual objects in the virtual 
world, including their constraints of motion, hierarchy, 
color, texture and other attributes must be made and entered 
JO into the virtual reality computer. Thus, it is desirable to 
make virtual world creation as simple as possible. 

One product used to create virtual worlds is the 
"Body Electric" program discussed in the above-identified 
patent application. The Body Electric program allows the 
5 user to create virtual world hierarchy trees and to couple 
data (real world or synthesized) to the tree nodes. Further 
details of the program maybe found in Appendix i. 

SUMMARY OF THE INVENTION 
0 The present invention is directed to an improved 

method and apparatus for creating virtual worlds wherein 
users may manipulate pictorial objects on a computer screen 
to effect complicated operations on the virtual world 
database. In one embodiment of the present invention, a 
5 computer display displays a data flow network as a plurality 
of interconnected units. An interactive program allows the 
appearance of the plurality of interconnected units to be 
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changed. The data flow network then is automatically 
altered to correspond with the visual changes. 

BRIEF DESCRIPTION OF THE DRAWINGS 
5 Fig. 1 is a diagram of a display screen which 

incorporates features according to the present invention; 
and 

Fig. 2 is a diagram of a display screen which 
incorporates interprocessor communication features according 
10 to the present invention* 

BRIEF DESCRIPTION OF THE APPENDICES 
Appendix 1 is a text description of the operation 
of a program that is used for creating virtual world 
15 databases; 

Appendix 2 is a listing of numerous examples of 
virtual worlds created with the present invention; 

Appendix 3 is a source code listing of "Body 
Electric" software which incorporates features of the 
20 present invention; and 

Appendix 4 is a source code listing of an 
interprocessor editor according to the present invention. 

DESCRIPTION OF SPECIFIC EMBODIMENTS 
25 Fig. 1 is a diagram showing a computer display 

showing features of the present invention. It should be 
referred to in the discussion which follows. 
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The Names Window 



The introduction of the Names window is one of the most fundamental 
changes of Body Electric 2.7b3. This window replaces many of the 
functions that the DM dialog box originally pexfomied. 

It might be useful to have Body Electric nxnning while reading this* Just 
Stan up BE 2.7b3 and open a T>Jew' file (from the Eilfi menu). Body 
Electric will ask you if you warn lo create a Tree or DM file. Choose DM. 



Below is the Names Window with the *type* box selected. This window 
is invoked by selecting •'Names" from the Window menu. 



The ^TYPfi" List. 
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The names you wmsee in ihislistaretheDM types that can beus^ la 
order to create a DM of any of these types, simply cUck on the name of the 
DM type yoa desire (without releasing the mouse button) and drag the DM 
onto a seleaed DM file window. Note that the DM ffle window should be 
selected (and not the Names window). 
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Th^ «ttaw" List 



raw EJEUniD 
EST1KB33HH 
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A listing of available raw inputs will appear when this box is checked. A 
raw input can be connected to a DM in a seleaed DM window by doing 



SUBSTITUTE SHEET 



wo 92/09948 



5 



per/ US9 1/08948 



5 



10 



basically the same process as was done to drag the DM out Click down 
on the raw input you need and drag it close to the DM input (on the left 
side of the DM) that you wish to connect it to. When (he raw input is 
within connecting distance, the input box it is closest to will light up. By 
releasing the mouse button* the raw input should connect itself to the DM. 
Another method of connecting a raw input to a DM is to click down on the 
DM input and then select the raw input needed. When the motise button is 
released, the connection should be inade« 



The ^Trgc" List 
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pitch 



roll 



When the tree box is selected, a listing of the tree nodes firom the Swivel 
file that is activated will appear. The tree file below is firom one of VPL*s 
worlds named ""Kitchen**. Tree node inputs and outputs can be conneaed 
to DM*s in the same fashion as raw inputs are. One thing that is slightly 
different is that when using tree nodes as inputs or outputs, the user must 
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specify which of the six degrees of fieedom is being IJiese six 

of freedom are x. y. z. yaw. pitch, and roU. (see also The 
Lower Six Options.") 



Names 



tree ESEQHH 



10 



15 



20 



roll 



ClockUall 
ClockFacft 
Sound2 
90Cleek 
eOCFock 
30Cloek 
ClockTwttlv^t 

ClockT«elvft2 
S«condHand 
ninuteHond 
HourKond 
Counterlop 
CounlarBoseboord 
Sink 

HoiGrob 

Ho IGrob^ounds. 
HolGrab-mln- 
HolCrob-nax. 
Co t dGrob 
Co i dGr ob^ounds^ 
ColdGrabUftin* 
ColdOrab^ox. 

FlowlngUaier 

Sound 4 



25 Nome! 

dm 
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When checked, this box makes ihe selected hsting appe^ in alphal»ttcal 
orto sS^S this is advantageous. When Ipoldng for a specific DM 
Stev^^ Othertimesan 
iSStetiSSg^ Forexample,ifyouweietrymg o 

^^S^^^ of a worid, alpha would picvcnt you from seeing this 
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The "dm" U5tmp 
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Ml 5 


6 



10 



Highlighting the dm box will show a list of the DMs that are in the active 
DM file. When you create a DM and name it , it will appear in this menu. 
With '<lm' conges the numbers 1 through 6 in the lower naif of the names 
menu section (see above). 

The dm menu item is useful as a time saver. Instead of searching through 
your file for a DM« you can find it here and use it as an input or output 
Just selea the in or out port (1-6) that you wish to use ana then use this 
list like you would use the **raw" and "trce"lists. 



The Lfl^cr Six Options 
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I pitch 



roll 
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The six bottom boxes in the Names Window menu are modifiers. 
Modifiers appear in three cases: with the Tree^ Type, and DM lists. These 
modifiers specify either the degree of freedom or the number of the input 
depending on the list chosen. For example, when creadng a set of DM*s 
that will move an object in VR, Body Electric usually needs to know what 
axis of the object should move* Do you want the object to move along the 
X axis, or do you w^nt it to roll? In order to use the correct "axis** of an 
objea, select x, y» z, yaw, pitch, or roll before ''grabbing'* the desired 
node. In the same way, choose the number of the input or output you are 
using. 



25 



The "Flex" Window 



30 



Body Electric 2Jb3 now allows the user to have more than one window 
open at a time. Because of this, the menu items "Flex** and Tree** firom 
the Windou^ menu no longer exist. In place of this. Body Electric now 
lists the names of the files that are open. To opena DM file (what used to 
be opened by **Flex** in the Windows menu) called rtame^ choose 
mzme.DM. To open a tree file (or the Swivel file), choose name. 
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15 



lUindouis 



stack lUini 
Tile lUfndo 
Close Rll U 
Rctluate 



Body Electric must know which tree and DM file to nm at any point in 
Sn2^ BtSuJe multiple files may be open simultaneously^ the user must 
S^B^lectrictowhichsetoffiles«b^^ rfo"* 
must mate sure thai the files being worted on are " Aaxvaied . To do 
SS^^peTthewindow to be activaed. With this window seleaed. choose 
"Activate" from the " ^dows menu. 

In Older to check whether a file is activated or not. look in the SQfldflas 
menu. Rles that are active have soUd diamonds to the left of *eir name. 
Ubraiies and Nested DM's Uiat belong to activated files have hoUow 
diamonds next to their names. 



J ^f. Wnrkhp nrh Tools 
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This is a picwre of a new DM window. On the left hand s^of^DM 
window are 6 tools. From top to bottom they are named the Selea tool, 
^H^^li^eOn/OttSy^idt tool, die Calibrate tool (it's a wrench), 
the Input/Output tool, and die DM descnption tooL 



The Select tool is the default tool for DM wmdows. It operates in the 
same fashion as the standard Macintosh arrow. To select a single box. 
Sot S to selea several DM's in the same area, pull a box out around 
Se desired boxes. To selea more than one DM one by one. hold down 
the shift button while clicking on die desired DM's. 
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With the Hand tool, the user can '"grab" the workspace within the DM 
window and slide it around like a sheet of paper on a desk. This is an 
ahemaiive to using the scroll bars. 



The On/Orr Switch tool allows the user to turn DM*s on or off very 
quickly. There are two ways of using this iooL One way is to select the 
On/Off tool and then click on the DM that needs to be turned on or off. 
Another way allows tlie user to turn on or off multiple DM*s 
simultaneously. First select the DM*s you want to change (see the Select 
Tool). Then hold down the mouse bunon on the On/0£f tool until it sians 
blinking* When you release the mouse button, the selected DM*s will 
either be turned on or off. 



The Scissors tool« is used to break connections that already exist 
between DM*s and other DM*s, Tree or Raw inputs. To use this tool 
select it and then click on the input pad you want to disconnecL The input 
that's going into that tab will be disconnected and the tab will go to it's 
default if it has a default or if not the DM will bt turned off. 



The Wrench, or Calibrate tool, is used to set the internal informadoo of 
a DM. For example, when calibrated, the Scale/Offset DM presents the 
dialog box below. Not all DM's have calibration boxes (e.^., 3Times, 
Average Two, Cube) 



FlKed ScQle/Offset DM module *UPL 
Research'. 



And the netu offset ualue: 



0.00000 



c 



OK 



[cencel] 



The Input/Output Tool lets the user modify the number and type of 
inputs and outputs for each DM. To use it, selea this tool and click on 
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any DM. Here is whai a common Input/Output dialog box looks like (this 
is from the DM Scale/Offset). 



10 
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20 
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30 



Scale/Offset 
Name: 



DMLines □ Display 
□ On 



undefined | 


Integer 


1 




1 






















outX ■ inX * const scale const offset. 



[ OK ) [ Cancel 



As you can see, this is also tlw dialog box in which you can name a DM. 
Widi the DMLines box checked lines are drawn from the inputs of this 
DM to the outputs of other DM's. This is the default. 

The Display check box 

When the On box is checked, the DM is on (fie., it is being looked at by 
Body Electric). 

The twelve boxes (just below the name box) control how many inputs and 
the outputs the DM will have, as well as what type each input and output 
will be. Say for example that you want to have ^ inputs on a Scale/Offset 
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10 



15 



20 



DM of type Integer. To do this, click down on the third box down in the 
left most column. The following menu will appear* 



Integer 

Long 

FlKed 

Real 

Point 

Same 

PointLUt 

undefined 



By selecting Tntcger" and releasing the mouse bunon, the integer inputs 
wUI now appear both in the dialog box as well as on the DM itself. The 
same methbd can be used to diange the type and number of the outputs. 

Note that by changing the inputs of a DM, the outputs usually are altered 
automatically based upon the DM being wotked on. 

And finally, the bottom-most box contains a description of the DM. 



By selecting the DM Description Tool and clicking on a DM, 
description of the DM will appear across the top of the wixulow. This box 
describes what the DM does. It is useful for finding out what a DM does 
as well as for double-checking what any q>ecific input is used for. 
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Another of the fundamental changes in Body Electric is the addition of 
Libraries and Nested DM's. 



Nested DM's are DM networks that exists inside other DM networks. 
When the outer netwoik executes, it calJs the Nested DM which executes 
it's network, and if it in turn has Nested DM's inside it they are called etc. 
This hierarchical descent occurs to as many levels as necessary. Each 
Nested DM has one input which detennines if the Nested DM will be fired 
at all and up to 5 other inputs which are passed to the DM network inside 
the Ne^ed DM. The Nested DM can have up to 6 ouq)uts that are passed 
from the network to the Nested DM. 



The Library DM is a special case of a Nested DM. In this case the DM 
network exists inside another DM file. This allows you to have fiinctions 
that are shared amongst different worlds.^ They can be added to any world 
by creating a Library DM that refers to the desiied library file. This DM 
was created to allow the user to include common and/or general Data 
Massage Modules into her or his program without too much grief. This 
DM has been created in the spirit of standard programming libraries such 
as the ones in C. 



Aflditional Features 



Find Dialog Box; 'This dialog box has a new checkbox called 'Global 
Search.* When it is unchecked all searches will be in the Tree or DM 
window you are currently editing. When it is checked the search will be 
conducted over all the currently opened files and for DMs all the nested 
DM's in the files. 



Mouse Active; There is a new menu item called Mouse Active under 
the DA menu. By default this comes up checked. When you uncheck it 
the mouse and keyboard items in the Raw Data Aixay become inactive. 
ITiat way you can use them as inputs to a DM network for testing. When 
you want to work on the network uncheck Mouse Active and your 
network won^ reaa to mouse movements as you are editing. 
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*■ Swivel IPC Specifications 

► Brief Descripcion 

► The Swivel IPC (Inter- Process Comuni cations) has been 

designed to allow external editors access to Swivel3D worlds 
whxle Swivel xs running, with it we have the ability to come 
up with new ways of creating and editing 30 solid objects 
Swivel can always draw any object as a list of polygons 'sut 



to edit an object Swivel looks at the objects type If it is 
10 ^Jt^'^^ swivel lathe object swivel calls it's internal 

10 editor but if it is another object type swivel trie"^ 

launch the associated editor and passes a message to edi 



15 



u«<B aB-ociacea eaicor ana passes a message to edit the 
given object. When you launch Swivel under Mullifindir it 
runs as one application (program) . Under MultiFinder there 
S^n th» ^'^^l ?f applications launched at the same time. 
SJo?ho^ onf^^^''*^.^ rr^i " °*»:'«<=c which uses 

another editor swivel looks up the name of the editor 

^SSi^*^"^""^ ^° launch! Chat, if successful swivel 

?he fdit.or.^^*H-? SwivlPC for that editor which teJls 

SllL^^e ori^lclt^"' ^^^-^ - updLIs°?^?r 

" "^PJ^*^ r^hffh^^e" Pf^"'* " through a resource called 
'5 J^ '^° "^^^ system file and is accessible to 

til applications. This resource doesn't normally exist so 
the first time SwivlPC is used the resource has io be added 
to tne system. 

► Since any application can access the message passing resource 
0 to protect our rights to the fechnoloS? 

Swivel will ignore messages from any editor until the editor 
has "registered-. This gets reset each time swivel- runs^ 
"Registering" means that the editor has to send a register 
message with a pointer to the string "Unauthorized use is 
subject to legal action." Until the editor does this swivel 
3 will ignore any messages from the editor. 

Message passing scenario 

" edito?'"(S6?)"''^^°^^ °" * ^'ii-riea by the terrain 

► Swivel tries to find and launch 'Terrain Editor' Swivel 
u also post a message to number 67 saying edit the object at 

location $xxxxx. .. 
" The Terrain Editor starts running and looks in the system 

resource for messages. It sees that there is a message from 
swivel to It to edit an object. It sends the registration 
message back to swivel and brings up a window so the- user can 
eaic tne object, 

► When the user is done. and has clicked OK the terrain editor 
sends a message back to swivel to check in the obiect the 
user has edited. 

J ► Svivel receives the message and replaces the old copy of the 

object With the newly edited object. 

► The terrain editor can at this point either quit or hang 
around waiting for some more messages from swivel 

Possible Messages 
MsCheckIn 

► Check an object into swivel that has been edited bv an 
editor. ^ 

► Data, DataLen - Handle to and length of memory in the 
editor for the new object. 

*■ modifiers : 
*- Ms Checkout 

► MsEditObj 

► MsGetCmtObj 
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► MsSetCmtObj 
c ► MsDoConnnand 

► MsQuiting 

► MsStaurtFast 

► MsEndFast 

► MsRegisterEd 

10 - message ^orm^^^^^ definition for System Resource 'YPJK' ,0 
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typeaex buitucc. v » . ^ * / 

short whomto; / whom is, the message for / 
short whomfrm; / whom is the message from / 

iSoS sta?u™ /' IPC status. Sending, received or 



short' m/sscount; /' Unique incrementing message 
number / 

short modifier- /' Modifier to the what field. V 
20 f^"^ ^Sen- /' TickCount when the message was sent, / 

ionj /* used by Individual messages however they 

want. / 

long DataLen; . , « 

\ QwivlPC SwivlPCPtr, SwxvlPCHandle ; 

sl^olll. ^en an?one reading the message must set it's 
e*-ai-tic to 0 thereby stopping the message. 
. whor^m " s the s^e J wholTto bit it • s whom the message xs 

Situs - The current state of the message passing mechanism. 

It is either: 

► 0 - not being used 

1 m^«ea~ has been sent and not received . 
40 > 1 ' message Jj^f ^ ^as posted an error, asxng 

incremented. • 
► what - The message verb. See aiJO^e- . , - 
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The purpose of the Swivel PC is to allow the Swivel 
interface to be customized and extended for specific purposes. 
Many different editor extensions are possible. The one 
implemented in the code provided in Appendix 4 is called the 
5 Orientation Editor. 

The Orientation Editor allows those aspects of the 
Swivel file used by Body Electric for the RB2 Virtual Reality 
System to be edited and stored within the Swivel file. These 
aspects include point attributes like bound, cube, line, 

10 LineDown, or Line Right, which determine how the point is 
rendered in Body Electric, as well as enabling particular 
degrees of freedom (Roll, Pitch, Yaw, X, Y or Z) to be output 
serially to another [host] computer. The Orientation Editor 
also allows the user to send command scripts to Swivel, to 

15 check all the names of objects in the Swivel file for 

uniqueness, to inherit selected attributes, and to mark, and 
return to world viewpoints. These capabilities enhance the 
user's ability to create Swivel Models for use in RB2 . 

TRhe improved Body Electric software described may 

20 be used for creating a number of virtual worlds. Examples of 
such virtual worlds are shown in Appendix 2, 

While the above is a complete description of a 
preferred embodiment of the present invention, various 
modifications may be employed. Consequently, the scope of the 

25 invention should not be limited except as described in the 
claims . 
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WHAT IS CIAIMED IS I 

1. An apparatus for creating virtual worlds 
comprising: 

5 display means for displaying a data flow network 

as a plurality of interconnected units; 

means for altering the appearance of the plurality 
of interconnected units on the display; and 

means for automatically altering the data flow 
10 network to correspond to the altered appearance of the 
plurality of interconnected units. 
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